Data compression is state of the art in permanent data storage (e.g. fixed disk, tape, diskette) as well as in data transfer (e.g. using a modem). There are well known methods for this, implemented either in software (e.g. as part of an operating system) or in hardware (e.g. chips in magnetic tape and data transfer devices). Although data can be sequentially compressed as a data stream, the better data compression methods require searching or sorting which complicates implementation in hardware and which makes the processing time indeterminate in software.
One example of a sorting method is the Burrows-Wheeler Transform (BWT) which processes a block of data as a single unit. A reversible transformation is applied to a block of text to form a new block having the same characters. This transformation tends to group like characters together so as to improve the probability of finding a character close to another instance of the same character. This sorted text can then be compressed with locally-adaptive algorithms, such as move-to-front coding in combination with Huffman or arithmetic coding.